For CRM and retention teams at iGaming operators and B2C fintechs
Would they have deposited anyway?
Independent, holdout-based measurement of what each CRM campaign earns in NGR, net of bonus cost. Computed in your own warehouse, not in the vendor's dashboard.
Customer.io today. Optimove, Smartico, Braze, Xtremepush, Fast Track next. No player data leaves your infrastructure.
How much of your bonus budget goes to players who would have deposited anyway?
A rough estimate from three numbers you already know. The exact figure needs a holdout — that is what we measure.
If you do not have a holdout, the second rate is a guess. Marketing audiences in our data deposit at roughly 8–18% in 14 days without any message.
Estimated bonus spend on non-incremental deposits
€150,000
About 75% of the budget is paid to players who were already going to deposit.
Formula: budget × (baseline rate ÷ campaign rate). An estimate, not a measurement. The measured figure depends on holdout size and window.
Measure it on one campaignCredited is not caused
Receiver conversion counts every deposit after a message, including the ones that would have happened without it. Active and VIP segments look best precisely because they deposit anyway.
The vendor grades its own homework
Built-in control groups report conversions, not revenue, and never subtract the bonus you paid. The platform that sends the message also decides whether it worked.
Three tools, three numbers
Multi-vendor stacks produce uplift figures that never add up to one answer: what did CRM contribute to NGR this month, after bonuses?
How it works
A measurement layer that sits beside your CRM, not inside it. Installed in your warehouse in 4–6 weeks.
- 01
Assign holdouts
5–10% of each campaign audience is held back, assigned deterministically in your warehouse before send and pushed to the CRM as one profile flag.
- 02
Join deliveries to money
Sends and clicks come back through the platform's warehouse sync and are linked to deposits, GGR and bonus cost per player.
- 03
Measure the difference
Sent versus holdout, intent-to-treat, with CUPED variance reduction, outlier trimming, a split-ratio check and confidence intervals.
- 04
Report in your KPIs
Incremental NGR, ROI, Bonus-to-GGR, by campaign, lifecycle flow and segment, with a verdict: works, no effect, loses money.
Proof without logos
We are early and do not list clients. Here is what you can check instead.
Nothing is a black box
- Deterministic holdout by salted hash, fixed before send, never reassigned
- Intent-to-treat: delivery failures cannot flatter the result
- CUPED on 6 weeks of pre-period, p99 trimming on the pooled sample
- Split-ratio check, Welch and bootstrap intervals, minimum detectable effect
- Self-excluded and restricted players excluded on both arms; cannot be switched off
What a result looks like
| Campaign | Deposit uplift | Net NGR / player | Verdict |
|---|---|---|---|
| Reactivation | +2.1 pp [0.9; 3.4] | +€0.70 | works |
| Reload bonus | +1.5 pp [0.4; 2.6] | −€3.69 | loses money |
| VIP weekly | +0.2 pp [−1.1; 1.5] | +€0.10 | no effect |
Illustrative numbers from our synthetic test stand, where the true effect is planted and recovered.
An operator-side data engineer
Built by a CRM data engineer who runs Customer.io and Smartico pipelines for a multi-brand, multi-licence operator: warehouse sync, money joins, segment reconciliation and the first holdout programme. Based in Poznań, working across EU and UK time zones.
Prefer to test us? Give us a campaign where you already know the answer. We will recover it blind.
Platforms
| Platform | Native control groups | What we add | Status |
|---|---|---|---|
| Customer.io | Holdout tests, conversion only | NGR net of bonuses, per flow and segment | Available |
| Optimove | Per-campaign, auto-shrinking control | Program-level, independent, fixed holdout | Next |
| Smartico / Xtremepush | Control %, conversion comparison | Revenue uplift and significance | Next |
| Braze / Fast Track | Global control (Braze); undocumented (Fast Track) | Cross-vendor single number | Roadmap |
Questions CRM teams ask first
Doesn't a holdout leave money on the table?
Only if the campaign works, and then the loss is 10% of one campaign for two weeks. If it does not work, the money was already being lost to the campaign, not to the holdout. A flat result is the first honest number your programme has produced.
Our audience is small. Will 10% be enough?
We compute the minimum detectable effect before the test starts. If 10% cannot see the effect you care about, we raise the share, extend the window, or measure the whole programme with a global holdout instead of one campaign.
Optimove / Braze / Customer.io already has control groups.
They report conversions and events, not NGR, and never subtract the bonus you paid. The control is also sized and read by the vendor that sends the message. We use their split when it is sound, and compute the money independently in your warehouse.
What about wagering requirements and bonus breakage?
We use realised bonus cost from your bonus tables, not the nominal amount. The gap between nominal and realised is reported separately, because it is where bonus abuse shows up.
Can we keep VIPs out of it?
Yes, as a segment with its own holdout share, including zero. The report then says what was measured and what was not. In practice VIPs are the segment most likely to deposit without the message, so they are also where bonus spend leaks most.
Does any data leave our warehouse?
No. Everything runs inside your own warehouse: ClickHouse today, BigQuery and Snowflake on request for your project. We need player id, timestamps, amounts and campaign ids, not PII. Access is read-only except for our own result tables.
Is this allowed under our licence conditions?
Holdouts apply to marketing messages only. Transactional and responsible-gambling messages are never withheld. Self-excluded, cooling-off and otherwise restricted players are excluded from both arms by a filter that cannot be disabled by configuration.
How long until the first number?
A retrospective audit of last quarter takes 2–3 weeks. A live holdout on one campaign gives a first read after 14 days and a stable one after 4–6 weeks.
What if the result is not significant?
Then the report says so, with the interval and the effect size you could have detected. We size tests to expect at least twice the minimum detectable effect, so this is rare, and when it happens it is still information.
Do you touch our sending or deliverability?
No. The only change in your CRM is one condition on the segment: do not send if the holdout flag is set. Content, timing and channels stay with your team.
Start small
Each step is useful on its own. None commits you to the next.
- 1Open calculator
Estimate your leakage
Three numbers, no form, above.
- 2Email me the note
Get the method note and sample report
Four pages, by email, no call.
- 3Book the review
Free 30-minute holdout design review
Bring one campaign. Leave with a test plan and its minimum detectable effect.
- 4Ask for a quote
Retrospective audit, fixed price
Last quarter's campaigns, bonus leakage by segment, 2–3 weeks. From €1,500.
- 5Discuss scope
Holdout implementation and monthly measurement
Assignment, CRM flag loop, models and dashboard in your warehouse. From €6,000, then from €750 a month.